Analyzing multidimensional process traces under edit-distance constraint

ABSTRACT

A method, system and computer program product for analyzing multidimensional data are disclosed. In embodiments, the method comprises obtaining an original set of data having a sequential order and multiple original dimensions; selecting a topic-based summarization scheme to summarize the original set of data; and applying the selected topic-based summarization scheme to the original set of data to transform the original set of data into a new set of data having fewer dimensions than the original set of data, while preserving, within a defined measure, the sequential order of the original set of data. In embodiments, the selecting a topic-based summarization scheme includes selecting a plurality of topics, each of the topic representing a set of the original dimensions. In embodiments, the applying the topic-based summarization scheme includes performing dimensionality reduction on the original set of data to transform the original dimensions to the topics.

BACKGROUND

This invention generally relates to analyzing multi-dimensional processtraces.

Many application domains, such as scientific workflows and businessprocesses, exhibit data models with complex relationships betweenobjects. These relationships may be represented as sequences, in whicheach data item can be annotated with multi-dimensional attributes.

SUMMARY

Embodiments of the invention provide a method, system and computerprogram product for analyzing multidimensional data. In embodiments, themethod comprises obtaining an original set of data having a sequentialorder and multiple original dimensions; selecting a topic-basedsummarization scheme to summarize the original set of data; and applyingthe selected topic-based summarization scheme to the original set ofdata to transform the original set of data into a new set of data havingfewer dimensions than the original set of data, while preserving, withina defined measure, the sequential order of the original set of data.

In embodiments, the selecting a topic-based summarization schemeincludes selecting a plurality of topics, each of the topic representinga set of the original dimensions.

In embodiments, the selecting a plurality of topics includes identifyinga plurality of attributes, each of the attributes having a number of theoriginal dimensions, and selecting the one of the attributes with thelargest number of the original dimensions.

In embodiments, the applying the topic-based summarization schemeincludes performing dimensionality reduction on the original set of datato transform the original dimensions to the topics.

As mentioned above, many application domains, such as scientificworkflows and business processes, exhibit data models with complexrelationships between objects. These relationships may be represented assequences, in which each data item can be annotated withmulti-dimensional attributes. In addition, there are increasing needsfor applications to analyze this data to get operational insights. Forexample, in business processes, users are interested in clusteringprocess traces into smaller subsets in order to discover process modelsthat are less complex in each subset.

To enable such applications, it is important to be able to measure thesimilarity between data objects. However, measuring the similaritybetween sequence-based data is a known computationally expensive task.

Embodiments of the invention provide a framework to perform efficientanalysis on sequence-based multi-dimensional data using intuitive anduser-controlled summarizations. Embodiments of the invention us an errormodel for summary-based similarity under an edit-distance constraint, asimilarity measure for sequences, and use summarization schemes thatprovide tunable trade-offs between the quality and efficiency ofanalysis tasks.

Embodiments of the invention provide a novel framework for analyzingmulti-dimensional process traces using intuitive and user-controlledsummarizations, and use an error model for summary-based similarityunder an edit-distance constraint.

Embodiments of the invention use summarization schemes that offerflexible trade-off between the quality and efficiency of analysis tasks.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 illustrates a loan application process and a sample trace.

FIG. 2 shows a sample trace of a semiconductor manufacturing process.

FIG. 3 shows an overview of an embodiment of the invention.

FIGS. 4(a), 4(b) and 4(c) show different forms of summarization of thetrace of FIG. 2.

FIG. 5 is a topic-based representation of the loan application processin FIG. 1.

FIG. 6 illustrates a topic summarization procedure in accordance with anembodiment of the invention.

FIGS. 7(a) and 7(b) show false positive rates by different summarizationschemes on similarity search task using a BPIC dataset.

FIGS. 8(a) and 8(b) show false positive rates by different summarizationschemes on similarity search task using a BANK dataset.

FIG. 9 shows the conformance fitness of clustering results on summaryspace by different summarization schemes.

FIG. 10 is a table that shows a traces clustering results structuralcomplexity comparison.

FIGS. 11(a) and 11(b) illustrate efficiency comparison between Randomand Topic summarizations using the same number of dimensions on the BPICand BANK datasets.

FIG. 12 depicts a cloud computing node that may be used in an embodimentof the present invention.

FIG. 13 depicts a cloud computing environment that may be used in anembodiment of the present invention.

FIG. 14 depicts abstraction model layers that may be used in anembodiment of the present invention.

DETAILED DESCRIPTION

Many application domains, such as business processes and scientificworkflows, exhibit data models in the form of multi-dimensionalsequences of objects. For example, in business processes, given anunderlying business process model represented as a directed acyclicgraph of activities, the traces generated from the execution of themodel are regarded as instances of the underlying model. Each tracecomprises a sequence of activities sorted by time, where each activityin the trace appears in the process model and may be repeated in thetrace. FIG. 1 shows an example of a loan application process model. Theactivities outlined in bold in the figure represent a possible executiontrace of the model. In addition to the sequential structure, eachactivity also contains multi-dimensional attributes. For example, anactivity in the loan application process can contain information aboutthe person and the department that are responsible for the activity, theperson who performs the activity, and the group to which he or shebelongs.

In another example, provenance data captured from the execution ofscientific workflows are also in the form of multi-dimensionalsequences. FIG. 2 shows a sample trace of a semiconductor manufacturingprocess, where each activity can include additional information, such asthe sector where the activity is performed and the person who isresponsible for that activity.

With the popularity of such applications, there are increasing needs toanalyze the data to get operational insights. For example, in businessprocess management, as business models discovered from a completeprocess traces are often complex and difficult to comprehend, users areinterested in clustering process traces into smaller subsets andapplying process discovery algorithms on each subset. The modelsdiscovered using only the traces in a cluster tend to be both lesscomplex and more accurate since there is less diversity among the traceswithin a cluster. In another example, scientists are interested inquerying provenance data of scientific workflow executions to look forprevious executions of a workflow that are similar to the one in thequery.

Analyzing the multi-dimensional sequence data, however, poses a numberof challenges. A first challenge is in terms of computational complexityof data analysis. For example, edit-distance is often used to capturethe similarity between sequences. However, since edit-distance isquadratic to the sequence length and each sequence can comprise hundredsof data items (e.g., in business processes), it is computationallyexpensive to compute the similarities between sequences. This isespecially challenging when dealing with large datasets and inapplications such as traces clustering, where a lot of similaritycomputations need to be calculated. The computational complexity canalso cause long application delay and thus affects interactiveapplications, such as similarity search, where users interact directlywith the application and expect to get the results in a timely manner. Asecond challenge is to combine multi-dimensional attributes of data withthe sequential structure between data objects into a unified approach.Edit-distance, for example, only concerns with counting the minimumnumber of basic operations required to transform one sequence ofactivities into the other, without considering the attributes ofactivities.

As illustrated in FIG. 3, embodiments of the invention provide aframework 300 for the efficient analysis of multi-dimensional sequencedata 302 under edit-distance constraint. Embodiments of the invention,analyze tasks that are based on edit-distance similarity measure, awidely used measure for sequences. Instead of performing computationallyexpensive analysis on the original high-dimensional data 302,embodiments of the invention transform the data into a summary spacethat has fewer dimensions, so that more efficient analysis can beapplied. To incorporate multi-dimensional attributes of data items intothe analysis, embodiments of the invention use summarization schemesthat allow users to select attributes as the summarization criteria. Inaddition to attribute-based summarizations, which produce summaries ofgood semantics but are limited in giving users control over theresolution of summaries (and thus, the efficiency), embodiments of theinvention use topic-based summarization that enables the flexibletrade-off between quality and efficiency of analysis tasks on summaries.In addition, embodiments of the invention use an error model for theedit-distance measure in the summary space to provide theoreticalguarantees for the results of analysis tasks on summaries.

The framework 300 of this invention is used for the analysis ofmulti-dimensional sequence data under edit-distance constraint, and FIG.3 shows a dataset 302 which may be analyzed in accordance withembodiments of the invention. Dataset 302 may be comprised of a set ofprocess traces or logs of scientific workflow executions. Running ananalysis, which would typically be computationally expensive due to thehigh-dimensionality of the data, provides results which are deemed asexact or “ground truth” answer 304.

Embodiments of the invention transform the original data into a newspace 306 with a smaller number of dimensions, thus avoiding thecomputationally expensive analysis on the original high-dimensionaldata. The resulting output 310, which is inherently different than the“ground truth”, is known as an approximate result. The summarization ofthe original data, and the analysis of the summarized data, presentchallenges.

One challenge is: how to generate summaries of data in a controlled andintuitive manner? Many sequence- and graph-based (in which sequence is aspecial form) summarization methods generate summaries using statistics,patterns, or sub-structures of the data. Thus, the resulted summariesare often difficult to interpret by users, as the resulted summarieslack the structural semantic connection with the originalrepresentation. The lack of structural semantics of summaries alsoprevents analysis tasks that rely on the structural information (such asedit distance-based analysis, whose results are easy-to-interpret byusers) to be performed on summary space. Also, under currently existingmethods, users do not have much control over how the summaries will begenerated. As a result, it is difficult to integrate user expertise andfeedback into the summarization process to guide the data analysis.

Another challenge is: How to relate the approximate results on thesummaries to the results on the original data? Since the analysisresults on data summaries might not be the same as those on originaldata, it is important to understand the relationship between the tworesults, and for all practical purposes, provide guarantees about thequality of results obtained from the summaries.

To address the above mentioned challenges, embodiments of the inventiondefine sequential-order-preserving summarization on sequences andintroduce several summarization schemes that give users more controlover the resulted summaries. Furthermore, embodiments of the inventionpresent an error model for summary-based similarity measure underedit-distance constraint. This error model provides quality guaranteeover the results of similarity search task. Empirical studies on realtraces show that the summary-based clustering of sequence data, inembodiments of the invention, produces results with comparable accuracy,compared with when using original data, while having better efficiency.

In embodiments of the invention, multidimensional sequence andsummarization of sequence can be defined, in general, as follows.

First, a multidimensional set of objects is defined:

-   Definition 1. (Multidimensional Set) A multidimensional set O is    defined as a set of objects O and a set of associated attributes    A=(A₁, A₂, . . . , A_(|A|)): O=    O, A    , each object o∈O is defined as a tuple: o=(A₁, (O), A₂(o), . . . ,    A_(|A|)(o)), in which each i-th dimension corresponds to the value    of attribute A_(i) of o, denoted as A_(i)(o).

Given the above definition of multidimensional set, multidimensionalsequences is defined:

-   Definition 2. (Multidimensional Sequence) A sequence p of size m on    a multidimensional set O is defined as an ordered set of m objects    in O :p=(p₁, p₂, . . . , p_(m)), p_(i)∈O, 1≤i≤m.

l_(p)(p) denotes the index, or position, of an object p in a sequence p.In the above definition, l_(p)(p_(i))=i, ∀1≤i≤m. For example, FIG. 2presents a sequence of objects defined on a multidimensional set withthree attributes: Activity, Sector, and Responsible.

In embodiments of the invention, different forms of summarization ofmultidimensional sequences are used to improve efficiency of sequenceanalysis. Before formally defining summarization of sequences, thenotion of many-to-one mapping of objects between multidimensional setsis defined:

-   Definition 3. (Many-to-one Mapping) A many-to-one mapping is defined    as an object mapping function ƒ from an original multidimensional    set O to a summary set S, ƒ:O→S, so that for each p∈O,    s∈S:s=ƒ(p).

Next, the summarization of sequences is defined based on many-to-onemapping ƒ, called ƒ-summarization:

-   Definition 4. (ƒ-Summarization) A ƒ-summarization of a sequence p on    O is defined as a summary sequence s on S, denoted as s=ƒ(p), where    each object p∈p is replaced by its many-to-one mapping ƒ:s=ƒ(p),    while retaining the same index l_(s)(s):=l_(p)(p).

A summarization of a sequence is said to preserve the sequentialrelationship from the original sequence if it satisfies the followingdefinition:

-   Definition 5. (Sequential Preserving Summarization) A    ƒ-summarization of a sequence p, denoted as s=ƒ(p), is a sequential    preserving summarization of p if: ∀p, p′∈p, if l_(p)(p)<l_(p)(p′),    then l_(s)(s)≤l_(s)(s′), with s=ƒ(p), s′=ƒ(p′).

ƒ-summarization preserves the sequential relationship since it retainsthe indices of objects in the original sequence onto the summarysequence. The ƒ-summarization defined in Definition 4, however, does notoffer any size reduction of the original sequences, which is vital inimproving the efficiency of sequence analysis. Therefore, reducedƒ-summarization, in which adjacent duplicate objects in the summarysequence are collapsed to reduce size of summarized sequence, isdefined.

Definition 6 (Reduced ƒ-Summarization) A reduced ƒ-summarization of asequence p on O is defined as a sequence s on S, denoted as s=ƒ*(p),where each object p∈p is replaced by its ƒ-based mapping s=ƒ(p) in sand, ∀p_(i), p_(i+1)∈p, 1≤i≤|p|−1, if p_(i)=p_(i+1), thenl_(s)(p_(i))=l_(p)(p_(i+1)).

Proposition 1. (Sequential Preservation of Reduced ƒ-Summarization) Areduced ƒ-summarization is a sequential preserving summarization.

Proof. Given an original sequence p=(p₁, p₂, . . . , p_(m)) on Op′=(p_(1′), p_(2′), . . . , p_(n′)) denotes a sequence on S and is thereduced ƒ-summarization of p. Elements in p′ can also be described asfollows: p_(1′)=p₁ and p_(i′)=ƒ(min_(l) _(p) _((·)){p_(j):p_(j)∈p, j≥i,p_(j)≠p_(i−1)}), for 1<i≤m (i.e., p_(i′) is the ƒ mapping of the firstnon-duplicate element since p_(i−1)).

Consider p_(i) and p_(j)∈p, 1≤i<j≤m. There are three possibilities:

-   -   p_(i)=p_(j) and p_(k)=p_(i)(∀k:i<k<j): In this case,        l_(p′)(p_(j))=l_(p′)(p_(i)) and l_(p′)(p_(k))=l_(p′)(p_(i)),        ∀k:i<k<j.    -   p_(i)=p_(j) and        k:i<k<j, p_(k)≠p_(i): In this case, l_(p′)(p_(i))<l_(p′)(p_(k))        and l_(p′)(p_(k))=l_(p′)(p_(j)).

As a result, l_(p′)(p_(i))<l_(p′)(p_(j)).

-   -   p_(i)≠p_(j): Since 1≤i<j≤m, l_(p′)(p_(i))<l_(p′)(p_(j))        according to the above definition of p′.

In all of the above cases, l_(p′)(p_(i))≤l_(p′)(p_(j)), and thus, p′preserves the sequential relationship between elements in p.

Summarization

Summarization schemes on multi-dimensional sequences are discussedbelow.

Attribute-Based Summarization

To incorporate multidimensional attributes of a sequence's data items,an attributes compatible mapping that leverages data item's attributesas summarization criteria, is defined as follows:

-   Definition 7. (Attributes Compatible Mapping) Given multidimensional    set O=    O, A    and a set of attributes A⊆A , a mapping ƒ is defined as A-compatible    mapping if: ∀p,q∈O, ƒ(p)=ƒ(q) if and only if A_(k)(p)=A_(k)(q),    ∀A_(k)∈A.

Next, an attribute-based summarization based on attributes compatiblemapping, is defined as follows:

-   Definition 8. (Attribute-based Summarization) Given a    multidimensional set O=    O, A    and a set of attributes A⊂A , an A-based summarization is defined as    a reduced ƒ-summarization, where the mapping ƒ is an A-compatible    mapping on O.

Although attribute compatible summarization provides a way for users tochoose attributes as summarization criteria and produces summaries thatare easy to interpret, it does not however give users much control overthe average length of summarized sequences, which is referred to asresolution. This is because attribute values are static and alreadydefined with original data. FIGS. 4(a)-4(c) show examples of differentattribute-based summarization of the trace in Example 2: Activity-based(FIG. 4(a)), Sector-based (FIG. 4(b)), and Responsible-based (FIG.4(c)). The Activity-based summary has highest resolution among theexamples, while the Responsible-based summary has lowest resolution(i.e., the most compact summary).

Since lengths of summarized sequences effectively influence theefficiency of sequence data analysis (i.e., the longer the lengths are,the more time-consuming data analysis will be), attribute-basedsummarization offers users with little flexibility in controlling thedesired efficiency of the data analysis on summarized sequences. On theother hand, it is often desirable that users are able to make trade-offdecisions between efficiency and accuracy of data analysis, especiallywhen dealing with large data, or data of high complexity. For example,in sequence similarity search application, users might decide totolerate a certain level of false positive in the results (e.g., 0.9false positive rate) as a trade-off for faster response (e.g., resultsare returned within 5 seconds).

To address the above challenge, embodiments of the invention use a novelsummarization scheme that offers more flexibility and better controlover the resolution of summaries, while still capturing semantic andsequential relationships of original data as with attribute-basedsummarization.

Topic-Based Summarization

Business processes, such as one in FIG. 1, can often be represented byhigher-level process models of fewer dimensions. FIG. 5 shows an exampleof a more general process model 500 of the process mode in FIG. 1.Process model 500 includes the activities of receive the application502, review the application 504, and inform the applicant of thedecision 506. Each activity in FIG. 5 corresponds to multiple activitiesin FIG. 1. Embodiments of the invention use a topic-based summarizationtechnique that captures the many-to-one mapping from original sequencesto topic-based summarization of fewer dimensions, where each topic is arepresentation of a set of original dimensions. Since the topics areimplicit from the original representation of sequences, dimensionalityreduction on original sequences is first performed to transform originaldimensions to topics, then, topic-based summarization is defined usingthe new representation.

Dimensionality Reduction on Sequences

Before applying dimension reduction techniques to original sequences, itis important to have an appropriate data representation for sequences.

In embodiments of the invention, an attribute of the original sequencesis selected, and multidimensional sequences are transformed to theappropriate attribute-based summarization. In embodiments of theinvention, the attribute with the highest number of dimensions isselected, as such the attribute often captures the most essentialinformation about the objects in the original multidimensional set. Forexample, in Example 2, Activity is the attribute with the highest numberof dimensions and it is also the base attribute to represent sequences,while other attributes, such as Sector and Responsible, providesupporting information for Activity.

In embodiments of the invention, each sequence p is represented as anumeric vector (ϑ₁, ϑ₂, . . . , ϑ_(|A*|)), where A* is the baseattribute that sequences are transformed to in the first step and |A*|is the number of dimensions on A*. ϑ_(i) is measured for p in a way thatcaptures both the local importance of each dimension and its specificityto a sequence. To capture the local importance, embodiments of theinvention use the frequency of the i-th dimension in p, denoted astf_(p) ^(i), that is defined by the number of items in p whose valuesequal the i-th dimension of A*, denoted as a_(i). To capture thespecificity, embodiments of the invention use the popularity of adimension across all sequences: df_(i)=|{p∈S|a_(i)∈p}|, where S is theset of all sequences. Intuitively, the higher df_(i) is, the morepopular the i-th dimension is and, thus, the less specificity it is to asequence. The formulation of ϑ_(i) is as follows:

$\begin{matrix}{\vartheta_{i} = \{ \begin{matrix}{( {1 + {\log ( {tf}_{p}^{i} )}} ) \times {\log ( \frac{S}{{df}_{i}} )}} & {{ifa}_{i} \in p} \\0 & {otherwise}\end{matrix} } & (1)\end{matrix}$

After representing sequences as vectors, the set of sequences S can berepresented as a matrix M, whose size is |S|×|A*| with each rowcorresponding to a vector representation of a sequence in S. With thismatrix representation, standard dimension reduction techniques can beapplied on M, such as non-negative matrix factorization (NMF), principlecomponent analysis (PCA), or singular value decomposition (SVD), amongothers. The results of these techniques can be presented as two matricesM′ and W. M, whose size equals |S|×k with k being the number of newdimensions (i.e., k=|S|), represents the original sequences on thesummary space. W, whose size equals |O|×k, represents the originaldimensions on the new dimensions, or topics (i.e., each row is a vectorrepresenting the distribution of an original dimension over the set ofnew dimensions).

Topic-Based Summarization From Dimensionality Reduction Results

In embodiments of the invention, based on the results of dimensionalityreduction, a many-to-one mapping is produced from the originaldimensions to topics. Two dimensions a_(i), a_(j) in the original spaceare likely to be in the same topic if their corresponding vectors in Whave high similarity (e.g., using Cosine similarity). In addition, a_(i)and a_(j) likely to be in the same topic if they frequently appear nextto each other in a sequence (i.e., they represent two closely relatedactivities in the underlining process model). From these insights, theproblem of a finding an optimal many-to-one mapping from originaldimensions to topics is modeled as a constrained optimization problem:

$\begin{matrix}{{{\underset{f}{argmax}\mspace{14mu} {\lambda \cdot {\sum\limits_{{f{(a_{i})}} = {f{(a_{j})}}}{\theta ( {a_{i},a_{j}} )}}}} + {( {1 - \lambda} ) \cdot {\sum\limits_{({a_{i},a_{j}})}{{\omega ( {a_{i},a_{j}} )}{\theta ( {a_{i},a_{j}} )}}}}} {{subjectto}\mspace{14mu} f\text{:}\mspace{14mu} O}arrow S {{\forall a_{i}},{a_{j} \in O},{{{iff}( a_{i} )} \neq {f( a_{j} )}},{{{thena}_{i} \neq {a_{j}.{S}}} = {k.}}}} & (2)\end{matrix}$

-   -   where θ(a_(i), a_(j)) represents the similarity between        dimensions a_(i) and a_(j) based on the their corresponding        representation in W , and ω(a_(i), a_(j)) represents the number        of times a_(i) and a_(j) appear next to each other in input        sequence set S. In embodiments of the invention, topic        summarization can be defined as follows:

Definition 9. (k-Topic Summarization) A k-topic summarization ofsequences from original multidimensional set O to a summary set S isdefined as a reduced ƒ-summarization, where the mapping ƒ is thesolution of the optimization problem defined in (2).

Proposition 2 (NP-hardness of k-Topic Summarization) The problem offinding a k-topic summarization of sequences on O is NP-hard.

Proof. The optimization problem in Equation 2 is a variant of the setpartitioning problem, and finding a feasible solution for such a problemis NP-hard.

Therefore, embodiments of the invention use a “greedy” heuristicapproach. In embodiments of the invention, a greedy approximationapproach is similar to that of agglomerative clustering algorithm.Specifically, the greedy approximation approach starts with treatingeach original dimension as a singleton cluster and then successivelymerges pairs of dimensions that are closest to each other until allclusters have been merged into a single cluster that contains alldimensions. This step creates a hierarchy where each leaf node is adimension and the root is the single cluster of the last merge. Becausein embodiments, a partition of disjoint k clusters is used as the newdimensions, the next step is to cut the hierarchy at some point toobtain the desirable number of clusters. To find the cut, an approach isused that is based on finding a minimum similarity threshold so that thedistance between any two dimensions in the same cluster is no more thanthat threshold, and no more than k clusters are formed.

The procedure of generating k-topic summarization of sequences is summedup in FIG. 6. This procedure comprises, as discussed above, generating avector representation of sequences 602, applying dimension reduction604, finding the optimal set of topics 606, applying semantic labels toidentified topics 610, and transforming the original sequences to thenew space 612. In this procedure, there are two steps that requireinputs from the user: the number of topics (i.e., dimensions) on thesummary space, and semantic labels for discovered topics. These inputscan be used by users to control the resolution of the summary space, aswell as to integrate user expertise into the summarization (and thus, tothe analysis tasks).

Error Model for Edit-Distance on Summaries

Embodiments of the invention focus on the relationship betweensimilarity of sequences on summary space with that on original spaceunder edit-distance constraint: ed(p,q) & ed(ƒ(p), ƒ(q)) (where ed isthe edit-distance function and ƒ is a summarization function).Edit-distance is selected as the similarity measure because it cancapture both the structural similarity (i.e., whether two sequencescomprise data items in similar orders) and content-based similarity(i.e., whether two sequences share a similar set of data items) betweensequences. In addition, edit-distance's result can be easily interpretedby users (i.e., the results can be presented as a chain of editoperators to transform a sequence to the other).

Edit-Distance

Edit-distance measure between two sequences can be formally defined asfollows:

Definition 10 (Edit-distance) Edit-distance between two sequences p,q,denoted as ed(p,q) is defined as the minimum number of edit operations(i.e., replace, delete, insert) that need to be taken to transform p toq.

In terms of the relationship between ed(p,q) and ed(ƒ(p),ƒ(q)),important properties include: contractive property and proximitypreservation property.

Definition 11. (Contractive Property) Given a summarization ƒ, theedit-distance measure satisfies the contractive property on ƒ ifed(p,q)≥ed(ƒ(p), ƒ(q)), ∀p q.

Contractive property is particularly important for applications such assimilarity search, because it guarantees that performing edit-distancebased similarity search on the summary space using ƒ will yield resultswith 100% recall. Specifically, given a query sequence p and anedit-distance threshold χ, the similarity search task needs to find allsequences in the sequence set S that have edit-distances with p smalleror equal than χ: S*={q∈S|ed(p,q)≤χ}. If the contractive property holdsfor a summarization ƒ, expensive calculation of edit-distance onoriginal space can be avoided by finding all sequences q that satisfythe threshold χ on summary space: S={q∈S|ed(ƒ(p), ƒ(q))≤χ}. Because ifed(p,q)≤χ, the ed(ƒ(p), ƒ(q))≤χ; guarantees that if q∈S*, then q∈S(i.e., 100% recall).

Definition 12. (Proximity Preservation Property) Given a summarizationƒ, edit-distance measure satisfies the proximity preservation propertyon ƒ if ed(p,q)≥ed(p,r), then ed(ƒ(p), ƒ(q))≥ed(ƒ(p), ƒ(r)), ∀p,q, r.

Proximity preservation property is particularly important forapplications such as traces clustering that groups similar traces intothe same cluster. This is because the proximity preservation propertyguarantees that traces that are similar in original space are alsosimilar in summary space. Thus, the clustering results on the summaryspace will likely be similar to those on the original space.

Error Model for Edit-Distance on Summary Space

To answer the questions regarding the relationship between ed(p,q) anded(ƒ(p), ƒ(q)), embodiments of the invention use an error model foredit-distance measure on summary space.

In this model, the contractive property does not hold in general foredit-distance between summarized sequences:

Theorem 1 Contractive property does not hold in general for summarizedsequences.

Proof. The above theorem is proven by providing a counter example.

Also, under certain circumstances, the contractive property holds foredit-distance between summarized sequences. One such circumstance iswhen the summarization ƒ is a non-reduced many-to-one summarization:

Theorem 2 If ƒ is a non-reduced many-to-one summarization on O, asdefined in Definition 4, then: ed(p,q)≥ed(ƒ(p), ƒ(q)), ∀p,q on O.

Proof. Assume that p=(p₁, p₂, . . . , p_(m)), q=(q₁, q₂, . . . , q_(n)).For compact representation, ed(p,q) is denoted as ed and ed(ƒ(p), ƒ(q))is denoted as ed′.

As part of the recursive Wagner-Fischer algorithm to calculateedit-distance between two sequences p and q, consider the step thatinvolves comparing two data items p_(i)∈p and q_(j) ∈q (1≤i≤m, 1≤j≤n).If the edit-distance at the current step is denoted as ed_(ij) anded′_(ij) (for edit-distance on summary space), based on the recursiveformula of the Wagner-Fischer algorithm:

If p_(i)=q_(j), then ed_(ij)=ed_(i−1, j−1). Because of the many-to-onesummarization ƒ, ƒ(p_(i))=ƒ(q_(j)). Hence, ed′_(ij)=ed′_(i−1, j−1). So,both ed_(ij) and ed′_(ij) do not require any edit cost in this case.

If p_(i)≠q_(j), then ed_(ij)=min(ed_(i−1, j)+1, ed_(i, j−1)+1,ed_(i−1, j−1)+1). Because of the many-to-one summarization ƒ, eitherƒ(p_(i))=ƒ(q_(j)) or ƒ(p_(i))≠ƒ(q_(j)). Thused′_(ij)=min(ed′_(i−1, j)+1, ed′_(i, j−1)+1, ed′_(i−1, j−1)+1) ifƒ(p_(i))≠ƒ(q_(j)) (i.e., one edit cost), or ed′_(ij)=ed′_(i−1, j−1) ifƒ(p_(i))=ƒ(q_(j)) (i.e., no edit cost). So, in this case, ed_(ij)requires one edit cost, while ed′_(ij) requires either one or zero editcost.

Therefore, eq_(ij)≥eq′_(ij), ∀i, j. Since the values {eq_(ij)} and{eq′_(ij)} form the matrix used by recursive algorithm to calculateed(p,q) and ed(ƒ(p), ƒ(q)) respectively, then ed(p,q)≥ed(ƒ(p), ƒ(q)).

Consider the case when ƒ is a reduced many-to-one summarization. In thiscase, even though it cannot be proven that the contractive propertyholds for any sequence pairs using summarization ƒ, less strict rulescan be derived to indicate whether the contractive property holds ordoes not hold for edit-distance of a particular pair of sequences p,qusing summarization ƒ:

Theorem 3 Given two sequences p,q on original space O, if ƒ is a reducedmany-to-one summarization on O, as defined in Definition 6, then:

-   -   If Γ_(p,q)≥Λ_(ƒ(p), ƒ(q)), then ed(p,q)≥ed(ƒ(p), ƒ(q)); or        edit-distance on summary space by ƒ satisfies the contractive        property.    -   If Γ_(ƒ(p), ƒ(q))>Λ_(p,q), then ed(p,q)<ed(ƒ(p), ƒ(q)); or        edit-distance on summary space by ƒ does not satisfy the        contractive property.

where Λ_(p,q)=max(|p|, |q|) and Γ_(p,q)=∥p|−|q∥, with |p| being thelength of p.

Proof. This theorem can be proven by noticing that Λ and Γ in factdefine the upper bound and lower bound on the edit-distance of a pair ofsequences.

The first rule is proven by using the chain rule of inequality:ed(p,q)≥Γ_(p,q)≥Λ_(ƒ(p), ƒ(q))≥ed(ƒ(p), ƒ(q)).

Similarly, for the second rule: ed(ƒ(p),ƒ(q))≥Γ_(ƒ(p), ƒ(q))>Λ_(p,q)≥ed(p,q).

Although Theorem 3 does not cover all cases, empirically it is shownthat the number of sequence pairs whose edit-distances on reducedsummarization violate the contractive property is very small. Thus, ithas a high recall for similarity search task when using reducedmany-to-one summarization.

For the proximity preservation property, even though a theoreticalguarantee cannot be obtained, it can be shown that the editdistance-based traces clustering results on summary space havecomparable accuracy, compared with those on original space, while havingbetter efficiency. This implies that the proximity relationship iswell-preserved on summary space under edit-distance constraint.

Evaluation

The utility of the framework of embodiments of the invention can bedemonstrated through an experimental evaluation of the effectiveness andefficiency of different disclosed schemes.

Evaluation Settings

Analysis tasks: The effectiveness and efficiency of the summarizationschemes are evaluated on two analysis tasks: trace similarity search andtraces clustering.

Datasets: Two datasets from different domains are used: a set of BPIC2015 dataset (1199 traces with 289 activity types) that contains processtraces of building permit applications, and a BANK dataset (2000 traceswith 113 activity types) that comprises synthetically generated logsthat represent a large bank transaction process.

Summarization schemes: Results of analysis tasks on summary space arecompared using the above-discussed summarization schemes (i.e. Topic andAttribute) Random summarization, which randomly maps an originaldimension to a new dimension in the summary space, and with the analysisresults on the original space.

Evaluation metrics: For the similarity search task, since thecontractive property holds for most of the cases, the recall rate ishigh. Thus, this example is only concerned with the false positive rateof the similarity search results. Given an edit distance threshold χ,this metric indicates that, out of all sequence pairs that satisfyed(ƒ(p), ƒ(q))≤χ on the summary space, how many of them actually satisfythe threshold in the original space: ed(p,q)≤χ.

For the traces clustering task, the clustering results are evaluatedusing process-specific metrics: weighted average conformance fitness,and weighted average structure complexity. While the process model'sconformance fitness quantifies the extent to which the discovered modelcan accurately reproduce the recorded traces, the structure complexityquantifies whether the clustering results produce process models thatare simple and compact. Given a summarization scheme, all sequences aretransformed to the summary space, and then traces clustering isperformed (using hierarchical clustering) with edit-distance as thesimilarity measure. Then, a process model is generated for each clusterusing a heuristic mining algorithm and then converted to the Petri-Netmodel for conformance analysis. Given the Petri-net model, two availableprocedures from the ProM framework are used for fitness and structuralcomplexity analysis: The Conformance Checker procedure is used tomeasure the fitness of the generated process models, and the Petri-NetComplexity Analysis procedure is used to analyze the structuralcomplexity of the process models. After fitness and complexity scoresare calculated for each cluster, the final scores are calculated as theaverage score over all clusters, weighted by clusters' sizes.

The evaluation was run on a computer with 16 GB of RAM and a 2.7 GHzquad-core Intel Core i7 CPU.

Effectiveness of Summarization Schemes on Similarity Search

To evaluate the effectiveness of different summarization schemes onsimilarity search task, the number of dimensions on summary space usedby Random and Topic was varied; and vary the attribute used by Attributewas varied and the false positive results over different edit-distancethresholds were compared.

FIGS. 7(a) and 7(b) show the results on the BPIC dataset, and FIGS. 8(a)and 8(b) show the results for the BANK dataset. As expected, the higherthe number of dimensions in the summary space (denoted by k) is, thebetter the result is (i.e., lower false positive rates). That isbecause, with a higher number of dimensions on the summary space,summaries of sequences more resemble the original sequences. Thus, thereis little difference between edit-distances on the summary space and onthe original space (hence, lower false positive rate). When comparingthe results of different summarization schemes on the same number ofdimensions, Random outperforms Topic summarization (at the cost ofinterpretability of summaries and the efficiency, as discussed below).For Attribute (FIG. 8c ), since there is no control over the number ofdimensions (since it depends on the attribute data), the quality of theresults also depend on the chosen attribute. Specifically, TrackedByattribute outperforms Sector and Tool. This is in part because there aremore dimensions on TrackedBy's summary space, and thus the summaries onTrackedBy space more resemble the original sequences. Sector and Toolproduce similar results, since similar Tools are often used in the sameSector.

Effectiveness of Summarization Schemes on Traces Clustering

For the traces clustering task, FIG. 9 shows that, surprisingly, usingsummarization schemes not only helps improve the efficiency of theclustering task (as discussed later), but also helps produce clusterswith process models of higher fitness, compared with the clusteringresults on original space. The trend is similar when varying the numberof clusters N. That is because measuring trace similarity on the summaryspace helps prevents noises that often exist when measuring similarityusing the original representation.

Among summarization schemes, Attribute helps produce clustering resultsof highest conformance fitness (especially when using TrackedByattribute). That is because Attribute summarizations capture better thesemantic relationship between traces (e.g., traces are similar if thecorresponding sequences of Sector, Tool, or TrackedBy are similar).

In terms of the structural complexity (FIG. 11), the Attributesummarizations again outperform other summarization schemes and theresult on original space. This is again due to the Attribute's abilityto capture semantic relationship between traces, and thus, it helpsproduce clusters whose process models capture actual groups of tracesthat share similar semantic (and thus, have simple model structure). Onthe other hand, Random is the worst performer, due to the fact thatrandom summarization could not capture the semantic relationship betweentraces.

In both conformance fitness and structural complexity tests, the Topicsummarization produces acceptable results. It again shows that Topic inembodiments of the invention, is the choice of approach when it comes toflexible trade-off between the effectiveness and efficiency of theanalysis task (i.e., similar to similarity search task, k can be tunedto achieve better efficiency for clustering task).

Efficiency of Summarization Schemes on Similarity Calculation

To evaluate efficiency of different summarization schemes, we vary thenumber of dimensions k on the summary space is varied and the time ittakes to calculate the edit-distance similarity between all pairs ofsequences is measured.

As expected, for both datasets and both Random and Topicsummarizations1, the higher k is, the longer the time it takes tocalculate the edit-distances (FIG. 11). That is because, the higher kis, the longer the sequences are in summary space, and thus, it is moreexpensive to calculate the edit-distances. At the same value of k, Topicoutperforms Random, which verifies Topic's ability to capture thesemantic relationship between original dimensions, and thussignificantly reduces the size of sequences on summary space, as well asthe processing time. Also, even at different values of k where similareffectiveness of results by Random and Topic are observed (e.g., k=2with Random and k=10 with Topic on Lithography dataset—See FIGS. 7(a),7(b), 8(a) and 8(b)), Topic is still much more efficient than Random.

Embodiments of the invention provide a framework to perform efficientanalysis on sequence-based multi-dimensional data using user-controlledsummarizations. Embodiments of the invention use a set of summarizationschemes that offer flexible trade-off between quality and efficiency ofanalysis tasks and an error model for summary-based similarity under anedit-distance constraint. The evaluation results on real-world datasetsverify the effectiveness and efficiency of the framework of embodimentsof the invention.

Embodiments of the invention may be used, for instance, for searching,clustering, and generating summaries. For example, given a trace,embodiments of the invention may be used to search for traces (whichexist in some data store) for traces that are similar to the giventrace. Also, given a set of traces, embodiments of the invention may beused to cluster similar traces together to detect anomalies or forfurther analysis. As another example, embodiments of the invention maybe used to generate representative summaries of traces for purposes ofillustrations.

It is understood in advance that although this disclosure includes adetailed description on cloud computing, implementation of the teachingsrecited herein are not limited to a cloud computing environment. Rather,embodiments of the present invention are capable of being implemented inconjunction with any other type of computing environment now known orlater developed.

Cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g. networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services)that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. This cloud model may includeat least five characteristics, at least three service models, and atleast four deployment models.

Characteristics are as follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider.

Broad network access: capabilities are available over a network andaccessed through standard mechanisms that promote use by heterogeneousthin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at some level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported providing transparency for both theprovider and consumer of the utilized service.

Service Models are as follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based e-mail).The consumer does not manage or control the underlying cloudinfrastructure including network, servers, operating systems, storage,or even individual application capabilities, with the possible exceptionof limited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises. Community cloud: the cloudinfrastructure is shared by several organizations and supports aspecific community that has shared concerns (e.g., mission, securityrequirements, policy, and compliance considerations). It may be managedby the organizations or a third party and may exist on-premises oroff-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting forload-balancing between clouds).

A cloud computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure comprising anetwork of interconnected nodes. Referring now to FIG. 12, a schematicof an example of a cloud computing node is shown. Cloud computing node10 is only one example of a suitable cloud computing node and is notintended to suggest any limitation as to the scope of use orfunctionality of embodiments of the invention described herein.Regardless, cloud computing node 10 is capable of being implementedand/or performing any of the functionality set forth hereinabove.

In cloud computing node 10 there is a computer system/server 12, whichis operational with numerous other general purpose or special purposecomputing system environments or configurations. Examples of well-knowncomputing systems, environments, and/or configurations that may besuitable for use with computer system/server 12 include, but are notlimited to, personal computer systems, server computer systems, thinclients, thick clients, hand-held or laptop devices, multiprocessorsystems, microprocessor-based systems, set top boxes, programmableconsumer electronics, network PCs, minicomputer systems, mainframecomputer systems, and distributed cloud computing environments thatinclude any of the above systems or devices, and the like.

Computer system/server 12 may be described in the general context ofcomputer system-executable instructions, such as program modules, beingexecuted by a computer system. Generally, program modules may includeroutines, programs, objects, components, logic, data structures, and soon that perform particular tasks or implement particular abstract datatypes. Computer system/server 12 may be practiced in distributed cloudcomputing environments where tasks are performed by remote processingdevices that are linked through a communications network. In adistributed cloud computing environment, program modules may be locatedin both local and remote computer system storage media including memorystorage devices.

As shown in FIG. 12, computer system/server 12 in cloud computing node10 is shown in the form of a general-purpose computing device. Thecomponents of computer system/server 12 may include, but are not limitedto, one or more processors or processing units 16, a system memory 28,and a bus 18 that couples various system components including systemmemory 28 to processor 16.

Bus 18 represents one or more of any of several types of bus structures,including a memory bus or memory controller, a peripheral bus, anaccelerated graphics port, and a processor or local bus using any of avariety of bus architectures. By way of example, and not limitation,such architectures include Industry Standard Architecture (ISA) bus,Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, VideoElectronics Standards Association (VESA) local bus, and PeripheralComponent Interconnects (PCI) bus.

Computer system/server 12 typically includes a variety of computersystem readable media. Such media may be any available media that isaccessible by computer system/server 12, and it includes both volatileand non-volatile media, removable and non-removable media.

System memory 28 can include computer system readable media in the formof volatile memory, such as random access memory (RAM) 30 and/or cachememory 32. Computer system/server 12 may further include otherremovable/non-removable, volatile/non-volatile computer system storagemedia. By way of example only, storage system 34 can be provided forreading from and writing to a non-removable, non-volatile magnetic media(not shown and typically called a “hard drive”). Although not shown, amagnetic disk drive for reading from and writing to a removable,non-volatile magnetic disk (e.g., a “floppy disk”), and an optical diskdrive for reading from or writing to a removable, non-volatile opticaldisk such as a CD-ROM, DVD-ROM or other optical media can be provided.In such instances, each can be connected to bus 18 by one or more datamedia interfaces. As will be further depicted and described below,memory 28 may include at least one program product having a set (e.g.,at least one) of program modules that are configured to carry out thefunctions of embodiments of the invention.

Program/utility 40, having a set (at least one) of program modules 42,may be stored in memory 28 by way of example, and not limitation, aswell as an operating system, one or more application programs, otherprogram modules, and program data. Each of the operating system, one ormore application programs, other program modules, and program data orsome combination thereof, may include an implementation of a networkingenvironment. Program modules 42 generally carry out the functions and/ormethodologies of embodiments of the invention as described herein.

Computer system/server 12 may also communicate with one or more externaldevices 14 such as a keyboard, a pointing device, a display 24, etc.;one or more devices that enable a user to interact with computersystem/server 12; and/or any devices (e.g., network card, modem, etc.)that enable computer system/server 12 to communicate with one or moreother computing devices. Such communication can occur via Input/Output(I/O) interfaces 22. Still yet, computer system/server 12 cancommunicate with one or more networks such as a local area network(LAN), a general wide area network (WAN), and/or a public network (e.g.,the Internet) via network adapter 20. As depicted, network adapter 20communicates with the other components of computer system/server 12 viabus 18. It should be understood that although not shown, other hardwareand/or software components could be used in conjunction with computersystem/server 12. Examples, include, but are not limited to: microcode,device drivers, redundant processing units, external disk drive arrays,RAID systems, tape drives, and data archival storage systems, etc.

Referring now to FIG. 13, illustrative cloud computing environment 50 isdepicted. As shown, cloud computing environment 50 comprises one or morecloud computing nodes 10 with which local computing devices used bycloud consumers, such as, for example, personal digital assistant (PDA)or cellular telephone 54A, desktop computer 54B, laptop computer 54C,and/or automobile computer system 54N may communicate. Nodes 10 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as Private, Community,Public, or Hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud computing environment 50 to offerinfrastructure, platforms and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 54A-N shownin FIG. 2 are intended to be illustrative only and that computing nodes10 and cloud computing environment 50 can communicate with any type ofcomputerized device over any type of network and/or network addressableconnection (e.g., using a web browser).

Referring now to FIG. 14, a set of functional abstraction layersprovided by cloud computing environment 50 (FIG. 13) is shown. It shouldbe understood in advance that the components, layers, and functionsshown in FIG. 14 are intended to be illustrative only and embodiments ofthe invention are not limited thereto.

As depicted, the following layers and corresponding functions areprovided:

Hardware and software layer 60 includes hardware and softwarecomponents. Examples of hardware components include mainframes 61; RISC(Reduced Instruction Set Computer) architecture based servers 62;servers 63; blade servers 64; storage devices 65; networks andnetworking components 66. In some embodiments, software componentsinclude network application server software 67 and database software 68.

Virtualization layer 70 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers71; virtual storage 72; virtual networks 73, including virtual privatenetworks; virtual applications and operating systems 74; and virtualclients 75.

In one example, management layer 80 may provide the functions describedbelow. Resource provisioning 81 provides dynamic procurement ofcomputing resources and other resources that are utilized to performtasks within the cloud computing environment. Metering and Pricing 82provide cost tracking as resources are utilized within the cloudcomputing environment, and billing or invoicing for consumption of theseresources. In one example, these resources may comprise applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.User portal 83 provides access to the cloud computing environment forconsumers and system administrators. Service level management 84provides cloud computing resource allocation and management such thatrequired service levels are met. Service Level Agreement (SLA) planningand fulfillment 85 provide pre-arrangement for, and procurement of,cloud computing resources for which a future requirement is anticipatedin accordance with an SLA.

Workloads layer 90 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation 91; software development and lifecycle management 92; virtualclassroom education delivery 93; data analytics processing 94;transaction processing 95; and interest highlight and analyzingmultidimensional process traces 96.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The description of the invention has been presented for purposes ofillustration and description, and is not intended to be exhaustive or tolimit the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope of the invention. The embodiments werechosen and described in order to explain the principles and applicationsof the invention, and to enable others of ordinary skill in the art tounderstand the invention. The invention may be implemented in variousembodiments with various modifications as are suited to a particularcontemplated use.

1. A method of analyzing multidimensional data, comprising: obtaining anoriginal set of data having a sequential order and multiple originaldimensions; selecting a topic-based summarization scheme to summarizethe original set of data; and applying the selected topic-basedsummarization scheme to the original set of data to transform theoriginal set of data into a new set of data having fewer dimensions thanthe original set of data, while preserving, within a defined measure,the sequential order of the original set of data.
 2. The methodaccording to claim 1, wherein the selecting a topic-based summarizationscheme includes selecting a plurality of topics, each of the topicrepresenting a set of the original dimensions.
 3. The method accordingto claim 2, wherein the selecting a plurality of topics includes:identifying a plurality of attributes, each of the attributes having anumber of the original dimensions; and selecting the one of theattributes with the largest number of the original dimensions.
 4. Themethod according to claim 2, wherein the applying the topic-basedsummarization scheme includes performing dimensionality reduction on theoriginal set of data to transform the original dimensions to the topics.5. The method according to claim 4, wherein the applying the topic basedsummarization scheme includes using the topic based summarization schemeto form the new set of data.
 6. The method according to claim 4, whereinthe performing dimensionality reduction on the original set of dataincludes producing a many-to-one mapping from the original dimensions tothe topics.
 7. The method according to claim 6, wherein the producing amany-to-one mapping from the original dimensions to the topics includestreating each of the original dimensions as a singleton cluster, andsuccessively merging pairs of the clusters.
 8. The method according toclaim 7, wherein the successively merging pairs of the clusters includessuccessively merging the pairs of the clusters until all clusters havebeen merged into a single cluster that contains all the originaldimensions to create a hierarchy including a multitude of leaf nodes anda root node, each of the leaf nodes representing one of the originaldimensions, and the root node representing said single cluster, and thehierarchy further including a plurality of levels between the leaf nodesand the root node, and each of the levels of the hierarchy including aplurality of clusters formed by merging the clusters at a higher levelof the hierarchy.
 9. The method according to claim 8, wherein theperforming dimensionality reduction on the original set of data furtherinclude cutting the hierarchy at one of the levels to obtain a selectednumber of the clusters.
 10. The method according to claim 9, wherein thecutting the hierarchy at one of the levels to obtain a selected numberof the clusters includes finding a minimum similarity threshold so thata distance between any two dimensions in the same cluster is no morethan said similarity threshold and no more than the selected number ofclusters are formed at said one of the levels.
 11. A system foranalyzing multidimensional data, comprising: one or more processors; anda memory coupled to the one or more processors; said one or moreprocessors configured for: obtaining an original set of data having asequential order and multiple original dimensions; selecting atopic-based summarization scheme to summarize the original set of data;and applying the selected topic-based summarization scheme to theoriginal set of data to transform the original set of data into a newset of data having fewer dimensions than the original set of data, whilepreserving, within a defined measure, the sequential order of theoriginal set of data.
 12. The system according to claim 11, wherein theselecting a topic-based summarization scheme includes selecting aplurality of topics, each of the topic representing a set of theoriginal dimensions.
 13. The system according to claim 12, wherein theselecting a plurality of topics includes: identifying a plurality ofattributes, each of the attributes having a number of the originaldimensions; and selecting the one of the attributes with the largestnumber of the original dimensions.
 14. The system according to claim 12,wherein the applying the topic-based summarization scheme includesperforming dimensionality reduction on the original set of data totransform the original dimensions to the topics.
 15. The systemaccording to claim 14, wherein the applying the topic basedsummarization scheme includes using the topic based summarization schemeto form the new set of data.
 16. A computer program product foranalyzing multidimensional data, the computer program product comprisinga computer readable storage medium having program instructions embodiedtherewith, the program instructions executable by a processor to causethe processor to: obtain, by the processor, an original set of datahaving a sequential order and multiple original dimensions; select, bythe processor, a topic-based summarization scheme to summarize theoriginal set of data; and apply, by the processor, the selectedtopic-based summarization scheme to the original set of data totransform the original set of data into a new set of data having fewerdimensions than the original set of data, while preserving, within adefined measure, the sequential order of the original set of data. 17.The computer program product according to claim 16, wherein the select atopic-based summarization scheme includes selecting a plurality oftopics, each of the topic representing a set of the original dimensions.18. The computer program product according to claim 17, wherein theselect a plurality of topics includes: identifying a plurality ofattributes, each of the attributes having a number of the originaldimensions; and selecting the one of the attributes with the largestnumber of the original dimensions.
 19. The computer program productaccording to claim 17, wherein the apply the topic-based summarizationscheme includes performing dimensionality reduction on the original setof data to transform the original dimensions to the topics.
 20. Thecomputer program product according to claim 19, wherein the apply thetopic based summarization scheme includes using the topic basedsummarization scheme to form the new set of data.